Expert-Based Software Measurement Data Analysis with Clustering Techniques

Software quality estimation models, used to predict the fault-proneness of software modules based on software metrics, are often constructed by training a classifier from labeled software metrics data. Two challenges often encountered in building an accurate model are the presence of “noisy” data and the possible unavailability of fault-proneness labels in real-world projects. The performance of a model often improves if outliers and noise are removed from the training data. More important, a classifier cannot be trained without fault-proneness labels. This article describes an exploratory analysis method that addresses these two challenges and that is built with clustering and the help of a software engineering expert. It is an unsupervised method since labeled training data are not required to predict the fault-proneness of software modules. We present two real-world case studies to verify the effectiveness of the clusteringand expert-based approach in predicting both the fault-proneness of software modules and potential “noisy” (e.g., mislabeled) modules. Keywords— Software Quality Estimation, Exploratory Data Analysis, Clustering, Noise Detection